Skip to content

Add pgTAP testing infrastructure #135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add pgTAP testing infrastructure #135

wants to merge 2 commits into from

Conversation

tobyhede
Copy link
Contributor

Summary

This PR introduces pgTAP as a robust testing framework for EQL, providing industry-standard PostgreSQL testing capabilities alongside the existing custom assertion helpers.

What's Included

Task 1: Install and Configure pgTAP ✅

  • Docker Integration: Created Dockerfile.pgtap that extends official PostgreSQL images (14-17) and installs pgTAP 1.3.5
  • Docker Compose: Updated docker-compose.yml to build custom images with pgTAP for all PostgreSQL versions
  • Installation Script: Added tests/install_pgtap.sql to install and verify the pgTAP extension
  • Test Runner: Created tasks/test-pgtap.sh - a mise task to run pgTAP tests with PostgreSQL version selection

Task 2: Structure Testing Suite ✅

Created comprehensive structure tests in tests/pgtap/structure/:

  • schema_test.sql (10 tests): Verifies eql_v2 schema, eql_v2_encrypted type, and eql_v2_configuration table structure
  • types_test.sql (7 tests): Verifies all index term types exist (blake3, hmac_256, bloom_filter, ORE variants)
  • functions_test.sql (9 tests): Verifies key functions exist with correct signatures (comparison, config management)
  • operators_test.sql (1 test): Verifies operators exist for eql_v2_encrypted type

All structure tests passing ✓

Task 3: Functionality Testing Suite (Partial) ✅

  • equality_test.sql (13 tests): Converted existing equality operator tests to pgTAP format
    • Tests eql_v2_encrypted = eql_v2_encrypted with HMAC and Blake3 indexes
    • Tests eql_v2.eq() function
    • Tests eql_v2_encrypted = jsonb and jsonb = eql_v2_encrypted operations
    • Includes both positive (matching) and negative (no match) test cases

Testing

All tests pass successfully:

mise run test  # Existing tests still pass

pgTAP tests can be run with:

mise run test-pgtap

Benefits

  • Industry Standard: pgTAP follows the TAP (Test Anything Protocol) standard used across the PostgreSQL ecosystem
  • Rich Assertions: Provides specialized assertions for schema testing, function signatures, and more
  • Better Organization: Clear separation between structure tests and functionality tests
  • CI Integration: TAP output format enables easy integration with CI/CD pipelines
  • Maintainability: Well-documented testing framework makes it easier for contributors

Implementation Plan

This PR implements Tasks 1-3 from docs/pgtap-implementation-plan.md. Remaining tasks:

  • Task 3.2-3.5: Additional functionality tests (comparison, JSONB operators, config, index terms)
  • Task 4: Data seeding patterns documentation
  • Task 5: CI pipeline integration
  • Task 6: Testing documentation and migration guides
  • Task 7: Advanced testing enhancements

Related

Based on the implementation plan in docs/pgtap-implementation-plan.md

@tobyhede
feat(test): add pgTAP testing infrastructure
5d40032 
Implement pgTAP test framework to provide industry-standard PostgreSQL testing capabilities alongside existing custom assertion helpers.

Changes:
- Add pgTAP to Docker containers (PostgreSQL 14-17)
- Create pgTAP test runner script (tasks/test-pgtap.sh)
- Add structure tests: schema, types, functions, operators
- Add functionality tests: equality operators
- Update docker-compose.yml to build custom images with pgTAP

Structure tests verify EQL schema elements exist with correct signatures.
Functionality tests validate encrypted data equality operations using pgTAP assertions.

This implements Tasks 1-3 from docs/pgtap-implementation-plan.md.
@tobyhede
feat(test): add comprehensive pgTAP functionality tests
36edf20 
Complete Task 3 from pgTAP implementation plan by adding functionality tests for:

- Comparison operators: <, <=, >, >=, <> with ORE indexes (12 tests)
- JSONB operators: ->, ->>, @>, <@ for encrypted data (13 tests)
- Configuration management: add/remove/modify search config (12 tests)
- Index terms: blake3, hmac_256, and ORE comparison functions (27 tests)

Total: 64 additional tests covering core EQL functionality

These tests validate:
- Encrypted data comparison operations work correctly
- JSONB extraction and containment operators function as expected
- Configuration management maintains state correctly
- Index term comparison functions return proper ordering results

All tests use pgTAP assertions (ok, is, cmp_ok, lives_ok, isnt_empty)
and follow the established pattern of setup/test/cleanup.
@tobyhede tobyhede mentioned this pull request Oct 22, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@tobyhede